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SUMMARY 


A  study  was  conducted  to  assess  whether  personality  measures  could  contribute  predictive 
utility  to  current  Air  Force  pilot  selection  procedures.  A  personality  inventory  was  given  to  a 
sample  of  509  United  States  Air  Force  pilot  candidates  before  entry  into  flying  training.  The 
inventory  was  designed  to  measure  five  dimensions  thought  to  be  associated  with  flying 
performance.  Of  the  five  characteristics,  three  measures  were  related  directly  with  training 
outcome:  hostility,  self-confidence  and  values  flexibility.  However,  incremental  validity  analyses 
did  not  provide  evidence  that  the  inventory  could  enhance  a  selection  model  that  combined 
currently  used  operational  test  scores  with  additional  measures  from  the  Basic  Attributes  Tests 
(BAT). 


PREFACE 


This  research  was  completed  underwork  Unit  77191845,  "Development  and  Validation  of 
Aircrew  Selection  Methodologies."  The  work  was  done  in  response  to  Request  for  Personnel 
Research  (RPR)  78-11,  "Selection  for  Pilot  Training." 

The  author  wishes  to  acknowledge  previous  efforts  on  this  research  project  by  several 
people.  The  inventory  would  not  have  existed  without  the  expertise  of  Dr.  John  Patterson  of 
the  Air  Force  School  of  Aerospace  Medicine  (SAM).  Preliminary  psychometric  analyses  were 
conducted  by  Dr.  Paul  Retzloff  while  he  was  at  the  School  of  Aerospace  Medicine.  Dr.  Edna 
Fiedler,  while  serving  as  a  summer  research  fellow,  provided  extensive  documentation  on 
initial  work  to  examine  the  predictive  utility  of  the  research  instrument.  Other  documentation 
and  data  analysis  support  were  provided  by  Edison  Watkins  of  the  OAO  Corporation. 
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PREDICTIVE  VALIDITY  OF  AN  AUTOMATED  PERSONALITY  INVENTORY 
FOR  AIR  FORCE  PILOT  SELECTION 


J.  INTRODUCTION 

Since  World  War  I,  psychological  tests  have  been  used  for  the  selection  of  military  aviators 
in  the  United  States  (North  &  Griffin,  1977).  Although  measures  of  personality  characteristics 
were  employed  in  World  War  I,  the  trend  during  and  since  World  War  II  has  been  toward  a 
greater  reliance  on  tests  of  psychomotor  coordination  and  cognitive  skills  (imhoff  &  Levine, 
1981).  This  trend  can  be  explained,  in  part,  by  the  observation  that  the  tests  developed  by 
the  Army  Air  Forces  in  the  1940s  laid  the  foundation  both  for  the  current  operational 
paper-and-pencil  test  used  by  the  United  States  Air  Force  (USAF)  and  for  an  experimental 
computer-administered  test  battery  recently  validated  for  pilot  selection  (Carretta,  1989).  Over 
the  years,  standard  procedures  have  evolved  for  measuring  "stick-and-rudder"  skills  and  cognitive 
abilities  such  as  information  processing  efficiency  and  spatial  visualization  skills.  In  contrast, 
no  such  standardization  exists  among  personality  measures.  Research  into  personality 
characteristics  and  their  relationship  to  aviation  performance  has  produced  a  substantial  body 
of  literature  but  has  not  clearly  identified  any  single  instrument  for  operational  use  as  an  aid 
to  military  pilot  selection  (Dolgin  &  Gibb,  1988). 

Several  factors  may  account  for  this  situation.  One  is  the  wide  variety  of  predictor  and 
criterion  measures  used  in  different  studies.  Much  of  the  earlier  research,  for  example,  focused 
on  the  relationship  between  clinical  scales  and  pilot  training  measures  (e.g.,  Sells,  1955),  whereas 
more  recent  studies  have  employed  scales  developed  for  "normal"  populations  (e.g.,  Jessup  & 
Jessup,  1971;  Novello  &  Youssef,  1974a,  1974b).  Also,  several  studies  have  focused  on 
differences  between  members  of  the  aviation  community  and  the  general  population  (e.g.,  Ashman 
&  Tefler,  1983;  Bartram  &  Dale,  1982;  Fry  &  Reinhardt,  1969).  Others  have  examined  the 
relationship  between  pilot  characteristics  and  safety  issues  (e.g.,  Sanders  &  Hoffman,  1975; 
Sanders,  Hoffman,  &  Neese,  1976;  Wichman  &  Ball,  1983). 

Another  factor  that  may  account  for  the  lack  of  a  cohesive  set  of  findings  concerns  a 
statistical  issue.  Many  studies  have  examined  only  simple  linear  relationships  between  a  large 
number  of  intercorrelated  personality  measures  and  some  criterion.  One  would  expect  that  the 
results  would  vary  somewhat  from  sample  to  sample-given  the  same  set  of  measures-simply 
as  a  consequence  of  multicollinearity  existing  among  the  predictor  measures.  Comparison  of 
studies  using  somewhat  different  instruments  therefore  becomes  all  the  more  complicated,  and 
rarely  has  the  practice  been  to  extract  more  global  measures  from  the  personality  instruments 
employed. 

Despite  this  lack  of  conclusive  research  findings  regarding  the  utility  of  personality  measures 
for  pilot  selection,  a  number  of  countries  currently  demonstrate  at  least  an  implicit  acknowledgment 
of  the  importance  of  personality  factors  for  mi''*ary  pilot  selection,  as  manifested  by  procedures 
that  involve  clinical  screening  and  interview  processes  (Jones,  1983).  Most  of  those  countries 
can  utilize  such  techniques,  requiring  intensive  screening  by  highly  trained  personnel,  because 
of  centralized  selection  procedures.  In  contrast,  pilot  candidates  for  the  USAF  are  drawn  from 
three  sources,  each  of  which  uses  somewhat  different  selection  procedures:  the  Air  Force 
Academy  (AFA),  the  Air  Force  Reserve  Officer  Training  Corps  (AFROTC),  and  the  Air  Force 
Officer  Training  School  (OTS).  Applicants  for  all  three  sources  are  dispersed  across  a  broad 
geographical  area  and  assignments  to  pilot  training  requirements  are  determined  by  a  number 
of  different  selection  boards;  hence,  the  need  exists  for  standardized  instruments  and  screening 
procedures  that  can  be  employed  in  a  decentralized  fashion. 
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In  response  to  continued  interest  in  techniques  to  improve  Air  Force  pilot  selection  procedures, 
the  present  research  was  designed  to  evaluate  the  potential  of  standardized  measures  of 
personality  for  reducing  attrition  and  enhancing  the  quality  of  pilot  trainees. 


II.  METHOD 


Sample 


The  sample  consisted  of  509  candidates  from  USAF  Undergraduate  Pilot  Training  (UPT). 
Most  of  the  sample  was  male  (99%;  N  =  503),  and  the  average  age  of  the  pilot  candidates 
was  23.8  years. 

Procedure 

Pilot  applicants  were  selected  into  the  Air  Force  on  the  basis  of,  in  part,  achieving  minimum 
scores  on  the  Air  Force  Officer  Qualifying  Test  (AFOOT).  The  AFOOT  is  a  paper-and-pencil 
aptitude  battery  that  includes  16  subtests  which  are  combined  into  five  composites.  Two  of 
these  composites  measure  aptitude  for  pilot  training:  a  Pilot  composite  and  a  Navigator-Technical 
composite  (Skinner  &  Ree,  1987).  Prior  to  entry  into  flying  training,  the  sample  was  administered 
the  Automated  Aircrew  Personality  Profiler  (AAPP),  the  instrument  used  in  the  present  investigation 
to  measure  personality  characteristics.  The  AAPP  was  administered  in  conjunction  with  an 
experimental  battery,  the  Basic  Attributes  Tests  (BAT).  The  BAT,  designed  to  measure  psychomotor 
coordination  and  information  processing,  was  administered  to  the  sample  subsequent  to  selection 
into  and  prior  to  Air  Force  jet  pilot  training. 

The  Undergraduate  Pilot  Training  (UPT)  program  in  which  respondents  participated  lasted 
49  weeks.  The  program  was  composed  of  two  phases  of  flight  training:  one  in  T-37  aircraft 
and  the  other  in  a  T-38  aircraft.  The  T-37  is  a  subsonic,  low-performance  jet  training  aircraft, 
whereas  the  T-38  in  a  high-performance,  supersonic  jet  aircraft.  Candidates  could  be  eliminated 
from  training  at  any  point  in  the  program  for  a  variety  of  reasons.  The  most  common  reasons 
for  elimination  were  flying  training  deficiency  (FTD),  medical  problems  (such  as  air-sickness), 
manifestation  of  apprehension  (MOA)  and  self-initiated  elimination  (SIE).  A  smaller  number  of 
candidates  were  eliminated  for  poor  academic  performance  or  insufficient  military  bearing. 

Candidates  remaining  in  the  program  after  41  weeks  of  UPT  were  considered  by  an  Advanced 
Training  Recommendation  Board  (ATRB)1  for  follow-on  instruction  in  either  a  fighter-attack- 
reconnaissance  training  track  (FAR)  or  a  tanker-transport-bomber  training  track  (TTB).  The  ATRB 
was  a  panel  of  T-38  Instructor  Pilots,  who  determined  whether  candidates  were  better  suited 
for  fast  jet  operations  (the  FAR  track)  or  for  flying  TTB  aircraft.  Actual  aircraft  assignment  was 
a  function  of  three  factors:  the  ATRB  outcome,  individual  preference,  and  aircraft  availability. 
Information  about  graduation  or  reason  for  elimination,  as  well  as  the  ATRB  recommendation, 
was  collected  at  the  end  of  UPT. 

The  number  of  respondents  having  data  for  each  of  the  predictor  and  criterion  measures 
is  shown  in  Table  1.  For  some  respondents  in  the  sample  tested  on  the  personality  measure, 
neither  training  outcome  data  nor  AFOOT  scores  were  available. 


’Use  of  the  ATRB  for  determining  follow-on  training  assignments  was  discontinued  subsequent 
to  UPT  class  89-09,  which  graduated  in  May  1989.  A  process  of  having  the  Wing  Commander 
make  follow-on  training  assignment  decisions  began  with  UPT  class  89-10. 
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Table  1.  Sample  Size  for  Predictor  and 
Criterion  Measures 


Measure 

Sample  size 

Basic  Attributes  Tests 

Item  Recognition 

509 

Encoding  Speed 

509 

Mental  Rotation 

321 

Time-Sharing 

509 

Self-Crediting  Work 

509 

Knowledge 

Activities  Interest  Inventory 
Automated  Aircrew 

509 

Personality  Test 

Air  Force  Officer 

292 

Qualifying  Test 

Undergraduate  Pilot  Training 

325 

Pass/Fail 

Instruments 


Air  Force  Officer  Qualifying  Test.  The  Air  Force  Officer  Qualifying  Test  (AFOOT)  is  a 
paper-and-pencil  test  comprised  of  16  subtests  measuring  such  factors  as  pilot  aptitude,  verbal 
ability,  and  math  knowledge  (Skinner  &  Ree.  1987).  The  Pilot  composite  consists  of  scores 
from  subtests  related  to  factors  such  as  verbal  ability,  aviation  knowledge,  and  scale  interpretation. 
The  Navigator-Technical  composite  contains  some  of  the  same  subtests  as  the  Pilot  composite 
but  includes  fewer  subtest  scores  related  to  verbal  aptitude  and  more  subtest  scores  related 
to  quantitative  skills  (see  Appendix). 

Basic  Attributes  Tests.  The  Basic  Attributes  Tests  (BAT)  battery  consists  of 
computer-administered  tests  measuring  characteristics  such  as  psychomotor  coordination, 
information  processing  skills,  time-sharing  ability,  and  personality  traits.  Twelve  measures  from 
six  tests  were  included  in  the  present  study.  For  three  information  processing  (item 

Recognition,  Encoding  Speed,  Mental  Rotation),  latency  and  accuracy  of  item  response  were 
recorded.  Latencies  were  also  recorded  for  the  other  three  tests,  as  were  a  measure  of  tracking 
(Time-Sharing),  a  measure  of  self-confidence  (Self-Crediting  Word  Knowledge),  and  a  measure 
of  risk-taking  proclivity  (Activities  Interest  Inventory).  (For  a  full  description  of  each  test,  see 
Carretta,  1989). 

Automated  Aircrew  Personality  Profiler.  The  Automated  Aircrew  Personality  Profiler  (AAPP) 
consists  of  202  items  representing  scales  from  several  instruments:  the  Minnesota  Multiphasic 
Personality  Inventory  (MMPI),  one  of  the  more  commonly  used  diagnostic  tools  in  clinical  practice 
(Dahlstrom,  Welsh,  &  Dahlstrom,  1972);  the  State-Trait  Anxiety  Inventory  (Bucky  &  Spielberger, 
1973);  the  Personal  Orientation  Inventory  (POI;  Knapp,  1975),  an  instrument  designed  to  assess 
an  individual's  aptitude  for  self-actualization;  the  Interpersonal  Behavior  Scale  (IBS;  Mauger  & 
Adkinson,  1980),  which  measures  assertive  and  aggressive  tendencies;  and  the  Jenkins  Activity 
Survey  (JAS;  Jenkins,  Rosenman,  &  Friedman,  1967),  designed  to  measure  personality  factors 
associated  with  chronic  heart  disease. 
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III.  RESULTS 


Descriptive  Statistics.  Descriptive  statistics  for  data  from  16  scales  of  the  AAPP  are  shown 
in  Table  2.  The  scale  scores  manifested  acceptable  variability,  and  each  was  normally  distributed 
There  were  no  cases  with  either  missing  data  or  extreme  scores. 


Table  2.  AAPP  Scale  Means  and 
Standard  Deviations  (N  =  509) 


Scale 

Mean 

SD 

Range 

Manifest  Hostility 

8.52 

2.56 

2  - 

15 

Acceptance  of  Aggression 

7.53 

1.59 

3  - 

12 

Frankness 

3.14 

1.12 

0  - 

5 

Need  for  Affection 

7.01 

1.90 

1  - 

1 1 

Denial  of  Social  Anxiety 

3.91 

1.80 

0  - 

6 

Social  Imperturbability 

8.16 

1.95 

0  - 

11 

Naivete 

4.95 

1.86 

0  - 

8 

Imperturbability 

4.33 

1.33 

0  - 

7 

Distrust 

6.10 

1  98 

1  - 

12 

Poignancy 

2.07 

1.14 

0  - 

6 

Brooding 

1.82 

1.27 

0  - 

7 

Ego  Inflation 

4.61 

1.50 

1  - 

9 

Values  Flexibility 

5.66 

1.76 

1  - 

1 1 

Amorality 

2.36 

1.13 

0  - 

6 

Hypomania 

9.72 

2  34 

4  - 

17 

Hard  Driving 

3.03 

98 

0  - 

5 

Factor  Analysis 


The  16  scale  scores  from  the  AAPP  were  factor  analyzed  using  principal  factoring  with 
oblique  rotation.  Five  factors  emerged  with  eigenvalues  greater  than  10  on  which  all  but  one 
scale  (Amorality)  manifested  factor  loadings  with  an  absolute  value  greater  than  30  (see  Table 
3). 


Based  on  the  exploratory  factor  analysis,  five  personality  scores  were  computed  Each  of 
the  five  measures  was  derived  in  accordance  with  the  factor  loadings  by  combining  the 
unweighted  raw  scores  for  each  of  the  component  scales.  Scales  with  negative  factor  loadings 
were  reverse  scored  before  being  summed  into  the  factor  scores.  Descriptive  statistics  for  the 
five  derived  measures  are  reported  in  Table  4,  and  their  intercorrelations  are  shown  in  Table  5. 
As  the  intercorrelations  demonstrate,  the  socially  desirable  factors  (self-confidence,  values 
flexibility)  were  independent  of  each  other  and  each  was  negatively  correlated  with  the  three 
less  socially  desirable  factors  (hostility,  depression,  mania).  The  latter  three  factors,  in  contrast, 
were  all  positively  correlated  with  one  another.  These  relationships,  then,  indicate  that  individuals 
who  were  either  self-confident  or  flexible  in  their  values  tended  not  to  be  hostile,  depressed  or 
manic.  Similarly,  individuals  who  reported  being  hostile  also  reported  high  levels  of  depression 
and  mania  as  well  as  low  levels  of  self-confidence  and  values  flexibility 
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Table  3.  Scale  Composition  of  Five  Personality  Factors 

(N  =  509) 


_ Factor _ 

Self-  Values  Depres- 

Scale  Hostility  confidence  flexibility  sion  Mania 


Manifest  Hostility . 69 

Need  for  Affection  . -68 

Naivete  . -92 

Distrust  . 68 

Ego  Inflation  . 35 

Frankness . 36 

Denial  of  Social  . 84 

Anxiety 

Social  Imperturbability . -99 

Imperturbability  . 55 

Acceptance  of . 66 

Aggression 

Values  Flexibility  . 64 

Poignancy .  57 

Brooding . 69 

Amoral  itya 

Hypomania .  67 

Hard  Driving . .  35 


Eigenvalues  4.69  2.1 5  1. 37  MG  1  02 

(Principal 
Components) 

%  of  Variance  29.3  13.3  8.6  6.9  6  5 

Explained 

Eigenvalues  4.30  1.81  .82  .60  42 

(Rotated 
Factors) 

%  of  Variance  26.9  11.3  5.1  3.8  2.6 

Explained 


aOnly  factor  loadings  with  absolute  value  >  .30  are  shown. 


Table  4.  Descriptive  Statistics  for 


Five  Personality  Factors  (N  =  509) 

Factor 

Mean 

SD 

Range 

Hostility 

33.06 

9.96 

9  -  63 

Self-confidence 

19.54 

4.98 

2  -  29 

Values  Flexibility 

13.20 

2.81 

6  -  21 

Depression 

3.89 

2.04 

0  -  11 

Mania 

12.75 

2.75 

6  -  21 

Table  5.  Intercorrelations  of 

Five  Personality  Factors  (N  =  509) 

Factor 

1. 

2. 

3. 

4. 

5. 

1.  Hostility 

__ 

2.  Self-confidence 

-32** 

- 

3  Values  Flexibility 

-24** 

04 

- 

4.  Depression 

40 

-33v  * 

-06 

- 

5.  Mania 

44** 

-11* 

-18** 

22** 

- 

Note.  Decimals  omitted. 

£  <  .05  that  variation  from  zero  correlation  is  due  to  chance. 
**  £  <  .01  that  variation  from  zero  correlation  is  due  to  chance. 


Relationship  Between  UPT  Pass/Fail  and  Personality  Factor  Scores 


The  point-biserial  correlations  between  the  five  personality  factors  and  UPT  training  outcome 
(pass/fail)  are  shown  in  Table  6.  As  these  data  indicate,  three  of  the  scales  were  associated 
with  UPT  outcome.  UPT  graduates-compared  to  those  who  did  not  complete  UPT-tended  to 
be  less  hostile,  more  self-confident,  and  more  flexible  in  their  values.  The  graduation  rate  was 
greater  than  80%,  so  that  the  p/q  split  for  this  sample  was  82/.18.  Thus,  the  maximum 
correlation  that  could  result  from  these  data  was  about  .7  (Gradstein,  1986). 

Incremental  Validity 


Analyses  reported  to  this  point  were  designed  to  address  the  question  of  whether  personality 
scores  derived  from  the  AAPP  demonstrated  a  relationship  with  UPT  training  outcomes.  From 
an  operational  perspective,  an  additional  question  also  needs  to  be  addressed:  Does  the  AAPP 
contribute  any  unique  predictive  utility  to  current  or  proposed  selection  models?  To  address 
this  question,  additional  data  were  examined  that  consisted  of  other  test  scores  from  an 
operational  instrument  (the  AFOQT)  and  from  an  experimental  test  battery  validated  for  pilot 
selection,  the  BAT.  The  experimental  tests  from  the  BAT  were  Item  Recognition,  Encoding 
Speed,  Mental  Rotation,  Time-Sharing,  Activities  Interest  Inventory,  and  Self-Crediting  Word 
Knowledge.  These  tests  provided  measures  of  speed  and  accuracy  of  information  processing, 
self-confidence,  risk-taking  propensity,  and  the  ability  to  perform  two  tasks  simultaneously. 
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Table  6.  Correlations  of  Five  Personality  Factor  Scores 
with  UPT  Outcome  (N  =  325) 


Personality  factor  score 

UPT  pass/fail 

Hostility 

-.12* 

Self-confidence 

.13** 

Values  Flexibility 

.12* 

Depression 

-.10 

Mania 

-.02 

*  £  <  .05  that  variation  from  zero  correlation  is  due  to  chance. 

**  £  <  .01  that  variation  from  zero  correlation  is  due  to  chance. 

To  examine  whether  the  AAPP  contributed  predictive  utility  over  and  above  that  provided 
by  the  AFOQT  and  BAT  scores,  a  multiple  regression  analysis  was  conducted.  The  criterion 
was  a  dichotomous  variable  representing  training  outcome  (graduate/non-graduate).  The  predictor 
set  consisted  of  scores  for  the  Pilot  and  Navigator-Technical  composites  of  the  AFOQT,  scores 
from  six  experimental  tests,  as  we"  as  five  scores  from  the  AAPP.  The  full  model  consisted  of 
19  predictors:  two  AFOQT  composites,  12  BAT  measures,  and  five  AAPP  scores. 

The  regression  analysis  indicated  that  the  full  model  was  significant  (R  =  33,  F[19,  261]  = 
1.69,  £  <  .05).  Elimination  of  the  five  personality  scores  did  not  significantly  reduce  the 
predictability  of  the  model  (R  =  .29,  F  change  [5,  261]  =  1.52,  £  <  .18).  Based  on  those 
results,  then,  it  appeared  that  the  AAPP  did  not  contribute  unique  predictive  validity  to  a  model 
incorporating  both  present  and  planned  selection  system  test  scores. 

The  above  analysis  provided  a  fairly  conservative  test  of  the  contribution  of  the  AAPP  to  a 
model  predictive  of  UPT  outcome,  as  the  reduced  model  included  two  tests  measuring  personality 
constructs  (Self-Crediting  Word  Knowledge  [SWK]  and  Activities  Interest  Inventory  [All]).  To 
provide  additional  information  about  the  capability  of  the  AAPP  to  contribute  to  a  selection 
system,  two  additional  regression  analyses  were  conducted.  In  one,  the  predictor  set  consisted 
of  the  full  model  as  described  above,  except  that  scores  from  Self-Crediting  Word  Knowledge 
were  removed  from  consideration;  the  second  full  model  was  constrained  even  further  with 
the  removal  of  scores  from  the  Activities  Interest  Inventory.  The  restricted  models  for  both 
analyses  removed  the  five  AAPP  scores  from  the  respective  full  models.  The  results  of  these 
analyses  paralleled  the  findings  reported  previously:  The  AAPP  did  not  add  significant  predictive 
variance  to  either  the  non-SWK  model  (F  change  [5,  268]  =  1.68,  £  <  .26)  or  the  non-SWK/AII 
model  (F  change  [5,  270]  =  1.42,  £  <  .22). 


IV.  CONCLUSIONS 

The  main  finding  from  the  present  study  was  that  the  use  of  self-report  personality  scores 
did  not  enhance  the  predictive  validity  of  a  selection  system  comprised  of  a  combination  of 
operational  tests  and  experimentally  validated  measures.  The  personality  measures  used  in  this 
study  demonstrated  a  weak  relationship  with  UPT  pass/fail,  which  suggests  that  there  is  little 
relationship  between  self-report  measures  and  pilot  performance,  as  reported  previously  (Dolgin 
&  Gibb,  1988).  Based  on  the  present  data,  future  research  might  benefit  more  from  utilization 
of  performance-based  personality  measures,  such  as  the  Self-Crediting  Word  Knowledge  test, 
than  on  the  use  of  self-report  personality  tests. 
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APPENDIX  A:  SUBTESTS  FROM  THE  AIR  FORCE 
OFFICER  QUALIFYING  TEST  (AFOOT) 


Subtest 

Composite 

Verbal 

Quantitative 

Navigator- 

Technical 

Pilot 

Verbal  Analogies 

X 

X 

Arithmetic  Reasoning 

X 

X 

Reading  Comprehension 

X 

Data  Interpretation 

X 

X 

Word  Knowledge 

X 

Math  Knowledge 

X 

X 

Mechanical  Comprehension 

X 

X 

Electrical  Maze 

X 

X 

Scale  Reading 

X 

X 

Instrument  Comprehension 

X 

Block  Counting 

X 

X 

Table  Reading 

X 

X 

Aviation  Information 

X 

Rotated  Blocks 

X 

General  Science 

X 

Hidden  Figures 

X 

Note.  A  fifth  composite,  Academic  Aptitude,  consists  of  all  subtests  from  the  Verbal 
and  Quantitative  composites. 
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